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DETAILED ACTION 
Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

2. Claims 1,3,4,5,6,7,9,10,12,13,14,15,16 are rejected under 35 U.S.C. 102(b) as 
being anticipated by Nakamura et al. (US Patent 6,005,987 A). 

Regarding claims 1 and 9, Nakamura et al. et al discloses an image processing 
method and apparatus (Fig. 2 is the apparatus.) for correcting image distortions 
(Differences in angles and parallax between images are corrected for as mentioned in 
col. 4, lines 39-44.) caused by oblique imaging in which an original image of an object 
on an object plane is taken from different oblique directions to obtain a plurality of 
partially overlapping images, comprising: 

a) a correspondence detecting unit (Fig. 2, num. 106:"PICTURE IMAGE 
POSITION CALCULATING UNIT" matches patterns with correlation values as - 
mentioned in col. 12, lines 34-40.) determining a feature point ((rx0,ry0) of figure 9A is a 
feature point or reference point of brightness values as mentioned in col. 12, lines 50-55 
and col. 13, lines 8,9.) of one of the plurality of partially overlapping images (Figure 9A 
has an image "l(k)" that is one of a plurality of images l(k) and l(k+1) for figures 9A and 
gBjespectively; that has the feature point.) corresponding to a common location of the 
original image, shared by the plurality of partially overlapping images (The "common 
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location" in Nakamura is identified in figures 9A and 9B by the boxed areas R1 and 
BM.), and determining a matched point of one of the other partially overlapping images 
corresponding to the feature point (The point (mx0,my0) of one of the other partially 
overlapping images l(k+1) of fig. 9B is a matched point in a matching region BM that is 
matched with the reference point (rx0,ry0) in the reference region R1 of the image l(k) 
fig. 9A as mentioned in col. 12, lines 34-55.) so that a direction of the object plane is 
calculated based on the feature point and the matched point (The direction of the object 
plane shown in fig. 9B as the region BM is calculated by subdividing the object plane as 
shown by the squares BM in fig. 9E, matching each subdivision with another subdivision 
using the feature and matched points from which a vector direction Vm of fig. 9E can be 
determined as mentioned in col. 13, lines 14-47.) 

b) a standard image setting unit (Figure 2, num. 108a:"JOINT LINE 
DETERMINING & PROCESSING UNIT" selects an image based on an optimum joint 
line as mentioned from col. 5, lines 58 to col. 6, line 13 and col. 22, lines 60-62.) 
selecting one of the plurality of partially overlapping images (The image of figure 13A is 
the one image selected either from aselection ofJmages from figures 1 3A and_13B_ 
which correspond with the pairs of partially overlapping images of figure 9A and 9B. The 
image of figure 13A is selected because the image of figure 13A has an optimum line 
selected from images 13A and 13B as shown in figures 13A-C and mentioned from col. 
5, line 58-col. 6, line 13 and col. 22, lines 34-62.) as a standard image whose image 
distortions are to be corrected (The standard image of figure 13A that was selected will 



Application/Control Number: 09/940,432 Page 4 

Art Unit: 2621 

prevent parallax or doubling of objects as distortion when constructing a composite or 
panoramic image as mentioned in col. 24, lines 43-52); and 

c) a distortion correcting unit generating a distortion-corrected image (Figure 
2, num. 109:"PICTURE IMAGE COMPOSITION & PROCESSING UNIT" generates a 
composite image that is distortion-corrected.) on a projection plane (Fig. 1 11, numerals 
803 and 804 shows a panoramic or composite image as the projection plane or output 
picture image region as mentioned in col. 18, lines 42-45.) by projecting the standard 
image onto the projection plane (The standard image of figure 13A is also shown as the 
image in figure 1 1G that is projected on the projection plane 803 and 804.) based on the 
direction of the object plane such that image distortions in the standard image are 
eliminated (The direction of the object plane BM of figure 9E was corrected beforehand 
(col. 13, lines 14-47) starting with the process of 106 of figure 2 so that parallax can 
then be corrected using the standard image 13A in the later process 108a of figure 2. 
The process of figure 109a of figure 2 uses the results of processes 106 and 108a to 
generate a distortion-corrected image free from misalignment and parallax as 
mentioned-in col. 4, lines 35-40.). . . _ .. ._. 
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Regarding claims 3 and 12, Nakamura et al. discloses the image processing 
method according to claim 1 wherein in said selecting step, one of the plurality of 
partially overlapping images is automatically selected as the standard based on a 
direction of a straight-line pattern (A vertical direction of a straight-line pattern L1(k) is 
shown in the image of figure 13A that is selected as the image with the optimum joint 
line that is selected from other lines as mentioned in regarding claims 1 and 9, above.) 
contained in each image (Figure 13A and 13B both are images that contains lines: L1(k) 
and L2(k) for the image of figure 1 3A and L3(k) for the image of figure 1 3B.). 

Regarding claims 4 and 13, Nakamura discloses the image processing method 
according to claim 1 wherein in said selecting step, one of the plurality of partially 
overlapping images is automatically selected as the standard (The image of figure 13A 
is automatically selected from the partially overlapping images using a computer 
program to execute the process of selecting a standard image as mentioned in col. 25, 
lines 59-64.) based on the feature point and the matched point (The standard image 
was selected using the processed result of the feature point (rx0,ry0) and matched point 

(mxCmyOJ as shown -by-the^series-of_procedures_mentiQned in col. 25. lines 59,60.) 

determined by said determining step. 
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Regarding claims 5 and 14, Nakamura discloses the image processing method 
according to claim 1, wherein in said selecting step, one of the plurality of partially 
overlapping images is automatically selected as the standard image (This portion was 
addressed in claim 4.) based on a calculated direction of the object plane for each of the 
partially overlapping images (The standard image of figure 13A was selected using the 
processed result of the calculated direction Vm of fig. 9E of the object plane BM of 
figure 9E as shown by the series of procedures mentioned in col. 25, lines 59,60.) 

Claim 6 is similar to claim 1 except for the additional element taught by 
Nakamura et al. of: 

Selecting one of the plurality of partially overlapping images as a standard image 
that contains a smallest amount of image distortions (The image l(k) of figure 1 1G which 
corresponds with the image of fig. 13A is selected as the standard image, because l(k) 
will prevent the least amount of parallax or object doubling using an optimum alignment 
or joining line (L sub. k) that was calculated by a minimizing function "a" out of all other 
joining lines from the plurality of partially overlapping images (Using figure 13A and 13B, 
-L1(k),L2(k) and-L3(k) are candidate joining lines within the respective plurality of 
overlapping images as mentioned in col. 22, lines 43-45 and line 51 shows the function 
"a". Note that the minimizing function "a" prevents parallax.) among the plurality of 
partially overlapping images (Figure 13A and 13B are a plurality of overlapping 
images.). Thus the image selected with the optimum joining line will prevent parallax or 
doubling of objects as distortion when constructing a composite or panoramic image as 
mentioned in col. 24, lines 43-52. 
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Claim 7 has similar claim language with claims 1 , 6 and 9 and has been 
addressed in claims 1 ,6 and 9. 

Regarding claim 10, figure 4 shows a plurality of imaging units or cameras 301 
and 302 that obtains an image of a scene in different directions 303 and 304, 
respectively. 

Claim 15 is similar to claim 1 except for the additional limitations which are 
discloses by Nakamura et al. of a computer-readable storage medium storing program 
code instructions for causing a computer to execute an image distortion correction 
processing (Nakamura et al. states, "The program stored in the storage medium is 
installed in the computer to cause the central processing unit to execute the program 
(col. 26, lines 23-25).") and code means (A series of program procedures or code 
means are mentioned from col. 25, line 59 to col. 26, line 15.), comprising: 

a) first program code means for causing the computer to determine a feature 
point etc. (A first program procedure calculates a position of an image using points at 
col. 26, lines 5,6 and addressed in claim 1.) . 

b) secondprogramxode„means_forxausing thejcomputerto select one of Jhe 

plurality of partially overlapping images as a standard image whose distortions are to be 
corrected (A second program procedure sets "the joint line on the profile of the object 
closest to the imaging means of the picture image"... as mentioned in col. 26, line 10- 
13); and 
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c) third program code means for causing the computer to generate a 
distortion-corrected image (A third program procedure forms "a composite panoramic 
picture image from the divided picture images." as mentioned in col. 26, lines 14, ISO- 
Claim 16 has been addressed claims 1,6 and 15. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 2,8 and 1 1 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nakamura et al.(US Patent 6,005,987 A) in view of lijima et al. (US Patent 
6,445,814 B2). 

Regarding claims 2,8 and 11, Nakamura teaches selecting a standard image 
using three methods (Nakamura et al., col. 22, lines 17-62) and selects objects using an 
area as shown in figure 9B by the area BM in relation to another area BL, but does not 
teach the limitations of claims 2 and 8. 

Regarding claims 2 and 1 1 , lijima et al. teaches in a selecting step, one of the 
plurality of partially overlapping images is automatically selected as the standard based 
on a ratio of an area of an object to an entire area of each image (A ratio between a 
background plane or entire area of each image as shown in fig. 2, num. 3 to an image 
sensing region or area of an object as shown in figure 2, num. 2 is used to select an 
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image from a plurality of images based on a reliability measure calculated from the ratio 
as mentioned in col. 22, lines 58-60 and col. 23, lines 36-40.), 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the teaching of the standard image selection of Nakamura et al. 
with the teaching of lijima et al.'s ratio, because lijima .et al's ratio provides a measure 
for "precisely obtaining] characteristic points (lijima et al., col. 22, lines 54-60)." 

Regarding claim 8, lijima et al. teaches a standard image setting unit (Fig. 2, 
num. 11 is an operation unit or standard image setting unit that allows a user to view an 
object or standard image from an arbitrary view point as mentioned in col. 23, lines 59- 
67.) that is configured such that a user is required to select the standard image when 
taking the original image from one of the oblique directions (A user views the standard 
image or object from the arbitrary view point to determine whether the object needs 
more processing as mentioned in col. 23, lines 64-67.), and wherein said image 
processing apparatus further comprises a notification unit which notifies the user that 
the standard image is currently taken (A display fig. 2, num. 8 is notification unit that 
visually-displays that-standardJmage-or object withjeliabilityjdata_as mentioned in col. 
23, lines 61-64.) 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the teaching of the standard image selection of Nakamura et al. 
with the teaching of lijima et al.'s user selection teaching, because lijima et al.'s user 
selection teaching allows the user "at a glance" or quickly determine whether the 
standard image needs further processing as mentioned in col. 23, lines 64-67. 
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Double Patenting 



5. Claims 1 ,6,7,9,15 and 16 are provisionally rejected under the judicially created 
doctrine of obviousness-type double patenting as being unpatentable over claims 4, 11 
and 18 of copending Application No. 09/645,51 1 in view of Nakamura et al. (US Patent 
6,005,987 A) 

Regarding claim 1 of the instant application, the copending application 
09/645,51 1 teaches in claim 4 all the limitations of claim 1 of the instant application * 
except for the step of selecting. 

However, the copending application of claim 4, lines 15,16 does suggest 
projecting one of two images in the step of generating. 

Nakamura et al. does teach selecting one of the plurality of partially overlapping 
images as a standard image whose image distortions are to be corrected. Nakamura 
selects one image (The image of fig. 13A) from a plurality (The images from fig. 13A 
and 13B) that are overlapping images as a standard image because the selected image 
will provide correction of parallax as mentioned in col. 24, lines 40-52. Note that the 
- selected image is selected based on an optimum joint line selected Jrom candidate lines 
within the images of figure 13A and 13B as mentioned in col. 22, lines 34-62.) 

Regarding claim 6 of the instant application, the copending application 
09/645,51 1 teaches in claim 4 all the limitations of claim 6 of the instant application 
except for the step of selecting. 

However, the copending application of claim 4, lines 15,16 does suggest 
projecting one of two images in the step of generating. 
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Nakamura et al. does teach selecting one of the plurality of partially overlapping 
images as a standard image that contains a smallest amount of image distortions 
among the plurality of partially overlapping images. Nakamura selects one image (The 
image of fig. 13A) from a plurality (The images from fig. 13A and 13B) that are 
overlapping images as a standard image because the selected image will provide 
correction of parallax (mentioned in col. 24, lines 40-52) that contains the fewest or 
smallest amount of parallax distortions as mentioned in Nakamura et al., col. 24, lines 
47-52. 

Regarding claims 7 and 9 of the instant application, the copending application 
09/645,51 1 teaches in claim 11 all the limitations of claims 7 and 9 of the instant 
application except for an apparatus comprising a standard image-setting unit 

However, the copending application of claim 11, lines 15,16 does suggest 
generating one of two images in the apparatus comprising a distortion correction 
means. 

Nakamura et al. does teach selecting one of the plurality of partially overlapping 

-images as a standard-image-that contains. a_smallest amount ofJmage distortions 

among the plurality of partially overlapping images as addressed in claim 6 whose 
image distortions are to be corrected as addressed in claim 1. 

Regarding claims 15 and 16 of the instant application, the copending application 
09/645,511 teaches in claim 18 all the limitations of claims 15 and 16 of the instant 
application except for the limitation "to select" of the second program means of claims 
15 and 16. 
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However, the copending application of claim 18, lines 19,20 does suggest 
generating one of two images in the fourth program means. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the- image-projecting step of copending Application No. 
09/645,51 1 with the teaching of Nakamura et al.'s teaching of selecting an image based 
on a optimum joint line, because Nakamura et al.'s image selection based on joint lines 
provides "...few portions having considerably different parallaxes [that] are jointed and 
the joint picture image looks more natural than in the conventional technology by which 
the quality is promoted (Nakamura et al. col. 24, lines 47-52)." 

This is a provisional obviousness-type double patenting rejection. 

Conclusion 

6. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

- -Peleg et al. (US Patent 6,532,036 B_1 ) is.pertinent as teaching a method of 
selecting an image for creating a panoramic image as mentioned in col. 8, lines 57-59. 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dennis Rosario-Vasquez whose telephone number is 
703-305-5431 . The examiner can normally be reached on 9-5. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo Boudreau can be reached on 703-305-4706. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




